To detect surface object information quickly and accurately by using remote sensing images is a current research hot spot. In order to solve the problems of inefficiency of the traditional manual visual interpretation segmentation method as well as the low accuracy and a lot of background noise of the existing remote sensing image segmentation based on deep learning in complex scenes, an image segmentation algorithm based on improved U-net network architecture and fully connected conditional random field was proposed. Firstly, a new network model was constructed by integrating VGG16 and U-net to effectively extract the features of remote sensing images with highly complex background. Then, by selecting the appropriate activation function and convolution method, the image segmentation accuracy was improved while the model prediction time was significantly reduced. Finally, on the basis of guaranteeing the segmentation accuracy, the segmentation result was further improved by using fully connected conditional random field. The simulation test on the standard dataset Potsdam provided by ISPRS showed that the accuracy, recall and the Mean Intersection over Union (MIoU) of the proposed algorithm were increased by 15.06 percentage points, 29.11 percentage points and 0.3662 respectively, and the Mean Absolute Error (MAE) of the algorithm was reduced by 0.02892 compared with those of U-net. Experimental results verify that the proposed algorithm is an effective and robust algorithm for extracting surface objects from remote sensing images.